70 research outputs found
Grasp Transfer based on Self-Aligning Implicit Representations of Local Surfaces
Objects we interact with and manipulate often share similar parts, such as
handles, that allow us to transfer our actions flexibly due to their shared
functionality. This work addresses the problem of transferring a grasp
experience or a demonstration to a novel object that shares shape similarities
with objects the robot has previously encountered. Existing approaches for
solving this problem are typically restricted to a specific object category or
a parametric shape. Our approach, however, can transfer grasps associated with
implicit models of local surfaces shared across object categories.
Specifically, we employ a single expert grasp demonstration to learn an
implicit local surface representation model from a small dataset of object
meshes. At inference time, this model is used to transfer grasps to novel
objects by identifying the most geometrically similar surfaces to the one on
which the expert grasp is demonstrated. Our model is trained entirely in
simulation and is evaluated on simulated and real-world objects that are not
seen during training. Evaluations indicate that grasp transfer to unseen object
categories using this approach can be successfully performed both in simulation
and real-world experiments. The simulation results also show that the proposed
approach leads to better spatial precision and grasp accuracy compared to a
baseline approach.Comment: Accepted by IEEE RAL. 8 pages, 6 figures, 3 table
Neural Field Movement Primitives for Joint Modelling of Scenes and Motions
This paper presents a novel Learning from Demonstration (LfD) method that
uses neural fields to learn new skills efficiently and accurately. It achieves
this by utilizing a shared embedding to learn both scene and motion
representations in a generative way. Our method smoothly maps each expert
demonstration to a scene-motion embedding and learns to model them without
requiring hand-crafted task parameters or large datasets. It achieves data
efficiency by enforcing scene and motion generation to be smooth with respect
to changes in the embedding space. At inference time, our method can retrieve
scene-motion embeddings using test time optimization, and generate precise
motion trajectories for novel scenes. The proposed method is versatile and can
employ images, 3D shapes, and any other scene representations that can be
modeled using neural fields. Additionally, it can generate both end-effector
positions and joint angle-based trajectories. Our method is evaluated on tasks
that require accurate motion trajectory generation, where the underlying task
parametrization is based on object positions and geometric scene changes.
Experimental results demonstrate that the proposed method outperforms the
baseline approaches and generalizes to novel scenes. Furthermore, in real-world
experiments, we show that our method can successfully model multi-valued
trajectories, it is robust to the distractor objects introduced at inference
time, and it can generate 6D motions.Comment: Accepted to IROS 2023. 8 pages, 7 figures, 2 tables. Project Page:
https://fzaero.github.io/NFMP
Bayesian Optimization-based Nonlinear Adaptive PID Controller Design for Robust Mobile Manipulation
In this paper, we propose to use a nonlinear adaptive PID controller to
regulate the joint variables of a mobile manipulator. The motion of the mobile
base forces undue disturbances on the joint controllers of the manipulator. In
designing a conventional PID controller, one should make a trade-off between
the performance and agility of the closed-loop system and its stability
margins. The proposed nonlinear adaptive PID controller provides a mechanism to
relax the need for such a compromise by adapting the gains according to the
magnitude of the error without expert tuning. Therefore, we can achieve agile
performance for the system while seeing damped overshoot in the output and
track the reference as close as possible, even in the presence of external
disturbances and uncertainties in the modeling of the system. We have employed
a Bayesian optimization approach to choose the parameters of a nonlinear
adaptive PID controller to achieve the best performance in tracking the
reference input and rejecting disturbances. The results demonstrate that a
well-designed nonlinear adaptive PID controller can effectively regulate a
mobile manipulator's joint variables while carrying an unspecified heavy load
and an abrupt base movement occurs
Grasp Transfer Based on Self-Aligning Implicit Representations of Local Surfaces
Objects we interact with and manipulate often share similar parts, such as handles, that allow us to transfer our actions flexibly due to their shared functionality. This work addresses the problem of transferring a grasp experience or a demonstration to a novel object that shares shape similarities with objects the robot has previously encountered. Existing approaches for solving this problem are typically restricted to a specific object category or a parametric shape. Our approach, however, can transfer grasps associated with implicit models of local surfaces shared across object categories. Specifically, we employ a single expert grasp demonstration to learn an implicit local surface representation model from a small dataset of object meshes. At inference time, this model is used to transfer grasps to novel objects by identifying the most geometrically similar surfaces to the one on which the expert grasp is demonstrated. Our model is trained entirely in simulation and is evaluated on simulated and real-world objects that are not seen during training. Evaluations indicate that grasp transfer to unseen object categories using this approach can be successfully performed both in simulation and real-world experiments. The simulation results also show that the proposed approach leads to better spatial precision and grasp accuracy compared to a baseline approach
Active Exploration Using Gaussian Random Fields and Gaussian Process Implicit Surfaces
In this work we study the problem of exploring surfaces and building compact
3D representations of the environment surrounding a robot through active
perception. We propose an online probabilistic framework that merges visual and
tactile measurements using Gaussian Random Field and Gaussian Process Implicit
Surfaces. The system investigates incomplete point clouds in order to find a
small set of regions of interest which are then physically explored with a
robotic arm equipped with tactile sensors. We show experimental results
obtained using a PrimeSense camera, a Kinova Jaco2 robotic arm and Optoforce
sensors on different scenarios. We then demonstrate how to use the online
framework for object detection and terrain classification.Comment: 8 pages, 6 figures, external contents (https://youtu.be/0-UlFRQT0JI
Simultaneous Tactile Exploration and Grasp Refinement for Unknown Objects
This paper addresses the problem of simultaneously exploring an unknown
object to model its shape, using tactile sensors on robotic fingers, while also
improving finger placement to optimise grasp stability. In many situations, a
robot will have only a partial camera view of the near side of an observed
object, for which the far side remains occluded. We show how an initial grasp
attempt, based on an initial guess of the overall object shape, yields tactile
glances of the far side of the object which enable the shape estimate and
consequently the successive grasps to be improved. We propose a grasp
exploration approach using a probabilistic representation of shape, based on
Gaussian Process Implicit Surfaces. This representation enables initial partial
vision data to be augmented with additional data from successive tactile
glances. This is combined with a probabilistic estimate of grasp quality to
refine grasp configurations. When choosing the next set of finger placements, a
bi-objective optimisation method is used to mutually maximise grasp quality and
improve shape representation during successive grasp attempts. Experimental
results show that the proposed approach yields stable grasp configurations more
efficiently than a baseline method, while also yielding improved shape estimate
of the grasped object.Comment: IEEE Robotics and Automation Letters. Preprint Version. Accepted
February, 202
Differentiable Robot Neural Distance Function for Adaptive Grasp Synthesis on a Unified Robotic Arm-Hand System
Grasping is a fundamental skill for robots to interact with their
environment. While grasp execution requires coordinated movement of the hand
and arm to achieve a collision-free and secure grip, many grasp synthesis
studies address arm and hand motion planning independently, leading to
potentially unreachable grasps in practical settings. The challenge of
determining integrated arm-hand configurations arises from its computational
complexity and high-dimensional nature. We address this challenge by presenting
a novel differentiable robot neural distance function. Our approach excels in
capturing intricate geometry across various joint configurations while
preserving differentiability. This innovative representation proves
instrumental in efficiently addressing downstream tasks with stringent contact
constraints. Leveraging this, we introduce an adaptive grasp synthesis
framework that exploits the full potential of the unified arm-hand system for
diverse grasping tasks. Our neural joint space distance function achieves an
84.7% error reduction compared to baseline methods. We validated our approaches
on a unified robotic arm-hand system that consists of a 7-DoF robot arm and a
16-DoF multi-fingered robotic hand. Results demonstrate that our approach
empowers this high-DoF system to generate and execute various arm-hand grasp
configurations that adapt to the size of the target objects while ensuring
whole-body movements to be collision-free.Comment: Under revie
Learning of Grasp Adaptation through Experience and Tactile Sensing
To perform robust grasping, a multi-fingered robotic hand should be able to adapt its grasping configuration, i.e., how the object is grasped, to maintain the stability of the grasp. Such a change of grasp configuration is called grasp adaptation and it depends on the controller, the employed sensory feedback and the type of uncertainties inherit to the problem. This paper proposes a grasp adaptation strategy to deal with uncertainties about physical properties of objects, such as the object weight and the friction at the contact points. Based on an object-level impedance controller, a grasp stability estimator is first learned in the object frame. Once a grasp is predicted to be unstable by the stability estimator, a grasp adaptation strategy is triggered according to the similarity between the new grasp and the training examples. Experimental results demonstrate that our method improves the grasping performance on novel objects with different physical properties from those used for training
Probabilistic consolidation of grasp experience
We present a probabilistic model for joint representation of several sensory modalities and action parameters in a robotic grasping scenario. Our non-linear probabilistic latent variable model encodes relationships between grasp-related parameters, learns the importance of features, and expresses confidence in estimates. The model learns associations between stable and unstable grasps that it experiences during an exploration phase. We demonstrate the applicability of the model for estimating grasp stability, correcting grasps, identifying objects based on tactile imprints and predicting tactile imprints from object-relative gripper poses. We performed experiments on a real platform with both known and novel objects, i.e., objects the robot trained with, and previously unseen objects. Grasp correction had a 75% success rate on known objects, and 73% on new objects. We compared our model to a traditional regression model that succeeded in correcting grasps in only 38% of cases
- …